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BACKGROUND 

1. Technical Field 

The present invention relates generally to computer- 
aided diagnosis (CADx) and, in particular, to a computer-aided 
diagnosis (CADx) method for aiding diagnosis of three- 
dimensional digital image data. 

2. Background Description 

Computer-assisted diagnosis is an important technology 
in many clinical applications. In current clinical practice, 
cancer or other diseases may be missed during a physician's 
un-aided examination of medical image data, in part because of 
the large volume of data. This is particularly a problem for 
screening applications, since there is generally little time 
to devote to the examination of each patient's data, and the 
entire range of the data must be examined to make sure it is 
free from disease. Computer analysis that is performed 
silently in the background can greatly aid physicians in their 
work. 

New technologies that offer three-dimensional (3D) 
scans of the human body, such as Magnetic Resonance Imaging 
(MRI) and Computed Tomography (CT) , offer tremendous 
opportunities for improved detection of disease. However, the 
change from two dimensions to three means that there is a much 




larger amount of data for the physician to examine. Thus, the 
assistance of computer analysis becomes even more important. 

The computer screen can only provide a two-dimensional 
(2D) view of a three-dimensional data set. There are many 
5 different approaches to presenting two-dimensional 

representations of such data, Traditional volume rendering 
algorithms sometimes produce satisfactory results, but they 
require pre-set thresholds to determine what types of tissue 
will be drawn as transparent and what types will be drawn as 
10 opaque. For complex anatomical structures, the volume 

_ rendering may be too cluttered for the physician to form a 

mental model of the underlying three-dimensional structure of 

til 

S the data, and how it corresponds to normal and abnormal 

U1 anatomical structure. It would be useful for a CADx system to 
lip provide a visualization that helps the physician quickly build 

O up the model, and easily distinguish abnormal structures from 

ry normal ones . 

i3 Unfortunately, many current CADx systems are not 

readily accepted by physicians, because their aid is seen as 

20 more of a distraction than a help. Many such systems present 

the results of the computer's diagnosis to the physician by 
marks, such as a red circle or arrow on the softcopy, which 
some physicians feel can create a bias in their interpretation 
of the data. Furthermore, many systems are perceived as a 

25 "black box"; that is, the physicians feel that they do not 

have any understanding of how the systems work and how they 
generate their diagnosis. 

It would be more desirable to integrate physician's 
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knowledge into the CADx process seamlessly, and without 
introducing marks onto the soft-copy that may be distracting 
or annoying to physicians. 



SUMMARY OF THE INVENTION 

The present invention is directed to a computer-aided 
diagnosis (CADx) method for aiding diagnosis of three- 
dimensional digital image data. The invention integrates a 
physician's knowledge into the computer-aided diagnosis 
process seamlessly, performs diagnostic computation 
accordingly, and then provides a novel presentation of the 
data that aids the physician in understanding the data. 

In a first aspect of the invention, there is provided a 
computer-assisted diagnosis method for assisting diagnosis of 
three-dimensional digital image data. The method includes the 
step of receiving indicia identifying at least one region of 
interest in a digital medical image. Three-dimensional 
objects within the at least one region of interest are 
identified. For a given three-dimensional (3D) object within 
the at least one region (ROD , an extent, a centroid, and a 
local spinning axis of the given object are determined. 
Moreover, a local spinning plane is determined for the given 
object, the local spinning plane being centered at the 
centroid and the local spinning axis. The local spinning 
plane is rotated at least a portion of 360 degrees. The 
rotating step includes the step of creating a view of the 
given object at predefined increments of rotation, so as to 
result in a plurality of views of the given object. 
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In a second aspect of the invention, the step of 
determining the extent of the given object includes the step 
of examining connected voxels within a predefined volume on 
adjacent tomographic slices . 
5 In a third aspect of the invention, the local spinning 

plane is initially oriented at a same angle as a current two- 
dimensional view of the three-dimensional digital image data. 

In a fourth aspect of the invention, the step of 
providing the plurality of views of the given object further 
10 includes the step of providing a plurality of views of areas 

m surrounding the given object. 

:Jf In a fifth aspect of the invention, the indicia is 

^ provided from a user through one of a mouse and an eye 
;£j tracking device. 

^ In a sixth aspect of the invention, the providing step 

□ further comprises the step of determining at least one of a 

ITU 

jtj volume, a geometrical location, and a center of mass of the 

Q given object. 

In a seventh aspect of the invention, the providing 
20 step further comprises the step of determining one of a 

circularity and a sphericity of the given object. 

In an eighth aspect of the invention, the providing 
step further comprises the step of determining a mean, a 
variance, and a min/max of intensity values within the given 
25 object. 

In a ninth aspect of the invention, the providing step 
further comprises the step of determining a texture, a surface 
smoothness, and regularity measures of the given object. 
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In a tenth aspect of the invention, the providing step 
further comprises the step of determining two-dimensional and 
three-dimensional shape measures of the given object. 

In an eleventh aspect of the invention, the method 
further includes the step of storing results from the 
providing step in a table for comparison with at least one of 
preceding or succeeding scans of a same patient. 

In a twelfth aspect of the invention, the method 
further includes the step of storing a confidence value in the 
table that indicates an estimate of a clinical relevance of 
the given object. 

In a thirteenth aspect of the invention, the method 
further includes the steps of setting thresholds for 
particular features of particular objects that represent 
whether the particular objects are abnormal, and identifying a 
given object that exceeds a given threshold. 

These and other aspects, features and advantages of the 
present invention will become apparent from the following 
detailed description of preferred embodiments, which is to be 
read in connection with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWING 

FIG. 1 is a block diagram of a computer processing 
system to which the present invention may be applied, 
according to an embodiment of the present invention; 

FIG. 2 is a flow diagram illustrating a computer-aided 
diagnosis (CADx) method for aiding diagnosis of three- 
dimensional digital image data, according to an illustrative 




embodiment of the present invention; 

FIG. 3 is a flow diagram illustrating step 235 of FIG. 
2 in further detail, according to an illustrative embodiment 
of the invention; 
5 FIG, 4 is a flow diagram illustrating a method for 

suggesting candidate areas of suspicion, according to an 
illustrative embodiment of the invention; 

FIG. 5 illustrates a two-dimensional "slice" from a 
three-dimensional data set obtained by computed tomography 
10 (CT) of the lungs, according to an illustrative embodiment of 

the invention; 

■:ff FIG. 6 is a magnified view of the region of interest 

U i 

y (ROI) of FIG. 5, according to an illustrative embodiment of 

U1 the invention; and 

IS* FIG. 7 illustrates frames from a "fly-around" movie, 

0 taken at 15 degree increments of the "spinning plane", 

|y according to an illustrative embodiment of the invention. 

^ DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

20 The present invention is directed to a computer-aided 

diagnosis (CADx) method for aiding diagnosis of three- 
dimensional digital image data. It is to be appreciated that 
the invention may be used to aid diagnosis of any abnormality 
in any part of the body which is represented in a digital 

25 image . 

FIG. 1 is a block diagram of a computer processing 
system to which the present invention may be applied, 
according to an embodiment of the present invention. The 




system 100 includes at least one processor (CPU) 102 
operatively coupled to other components via a system bus 104. 
A read only memory (ROM) 106, a random access memory (RAM) 
108, a display adapter 110, an I/O adapter 112, and a user 
5 interface adapter 114 are operatively coupled to system bus 

104. 

A display device 116 is operatively coupled to system 
bus 104 by display adapter 110. A disk storage device (e.g., 
a magnetic or optical disk storage device) 118 is operatively 
10 couple to system bus 104 by I/O adapter 112. 

^ A mouse 120 and eye tracking device 122 are operatively 

coupled to system bus 104 by user interface adapter 114. The 
Q mouse 120 and eye tracking device 122 are used to aid in the 
III selection of suspicious regions in a digital medical image. A 
1J5 keyboard 124 may also be operatively coupled to system bus 104 
Q by user interface adapter 114. The keyboard may be used to 
nj input and. output information to and from CAD system 100. 
Q The CAD system 100 may also include a digitizer 126 

operatively coupled to system bus 104 by user interface 
20 adapter 114 for digitizing a developed x-ray. Alternatively, 

digitizer 126 may be omitted, in which case a digital image 
may be input to system 100 from a network via a communications 
adapter 128 operatively coupled to system bus 104. 

FIG. 2 is a flow diagram illustrating a computer- 
25 assisted diagnosis method for assisting diagnosis of three- 

dimensional digital image data, according to an illustrative 
embodiment of the present invention. 

Indicia identifying a region of interest in a digital 



10 



20 



25 



r 



medical image is received (step 205) . Step 205 is also 
referred to herein as "click-on" or "focus-on". In this step, 
the physician examines two-dimensional views of the three- 
dimensional data in the usual clinical way. The physician 
then defines the region of interest (ROI) , by either clicking 
on the suspicious region using mouse 12 0, or by using other 
devices such as, for example, wireless eye- tracking device 
122. The ROI may also be automatically detected and suggested 
by a computer method. It is to be appreciated that the 
preceding examples of how a region of interest is identified 
are merely illustrative and, thus, other ways of accomplishing 
the same may be readily contemplated by one of ordinary skill 
J in the related art, while maintaining the spirit and scope of 
\-^the invention. 

Three-dimensional objects within the region of interest 
are automatically identified (step 210). Step 210 is also 
referred to herein as " center- in" . In this step, once the ROI 
is selected by the physician (at step 205) , three-dimensional 
objects are automatically found, if present, within the ROI. 
This object is known as the target object (or the given 
object) . 

To automatically find a target object, the volume of 
the medical image data immediately surrounding and including 
the region of interest (ROI) is searched according to standard 
search techniques. The extent of the search is defined by a 
threshold that is adaptive to its local maximum intensity 
value within the ROI. An object is detected when a voxel in 
the search area has a value which indicates that it 
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corresponds to tissue, such as blood, fat, or bone, rather 
than air. Other 3D morphological operations can also be used 
to find the 3D object. Given the teachings of the present 
invention provided herein, one of ordinary skill in the 
rele^^ 

target objects can be automatically identified, while 



maintain! 



ming th^spir^t— and^s^ 

'"Fo-r a given (three-dimensional) target object within 
the region of interest, an extent, a centroid, and a local 
spinning axis of the target object are automatically 
determined (step 215) . The extent of the target object is 
determined, by examining connected voxels within a predefined 
volume on the adjacent topographic slices. 

Steps 220 through 245 are referred to herein as "fly 
around". For these steps, upon the physician's request for 
further examination of the target object, the three- 
dimensional volume of data surrounding the object is displayed 
for the physician to simulate how it would appear if the 
doctor could look all the way around the object. 

At step 220, a local spinning plane is determined for 
the target object. The local spinning plane is centered at 
the centroid and the local spinning axis determined at step 
215. The local spinning plane shows a small window of detail 
around the target object. 

The local spinning plane is initially oriented at the 
same angle as the current two-dimensional view of the whole 
data set (step 225) . The local spinning plane is then rotated 
(spun) 360 degrees (or a portion thereof) (step 230) . During 
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the spinning of • the local spinning plane, a view of the target 
object is created at predefined increments of rotation, so as 
to result in a plurality of views of the target object (step 
235) . Moreover, during the spinning of the local spinning 
plane, diagnostic information is computed about the target 
object for each of the different views (step 240) . 

The different views are displayed in sequence, giving 
rise to a short "movie" or cine loop that simulates what it 
would look like to fly around the object (step 245) . It is to 
be appreciated that, at step 240, the views may be stored in 
place of, or in addition to, displaying the views to the 
physician. If the views are stored, then they may be later 
retrieved for subsequent viewing and/or analysis. 

The advantage of the "spinning plane" view is that it 
allows a useful visualization of the three-dimensional data 
within the constraint of the two-dimensional computer display. 
This visualization is more like the two-dimensional 
pathological cutting image "slices" with which physicians are 
already familiar . 

In a single two-dimensional projection, the underlying 
three-dimensional appearance of many objects is ambiguous. A 
spherically shaped object, such as a nodule, will appear as a 
circle in two-dimensional projection. However, a cylindrical 
object, such as a blood vessel, can also appear as a circle, 
if it is aligned perpendicular to the current two-dimensional 
projection. The spinning plane of the invention presents a 
multitude of views in a fast and comprehensible way, which 
allows the view to easily distinguish between scenarios that 




were initially ambiguous. 

This "spinning plane" view is quite different than the 
typical volume rendering that many systems use to project 
three-dimensional data. These volume renderings require that 
5 a lot of decisions be made in advance as to what pixel values 

should appear transparent and what values should appear opaque 
or semi-opaque. 

FIG. 3 is a flow diagram primarily illustrating step 
235 of FIG. 2 in further detail, according to an illustrative 
10 embodiment of the invention. That is, most of the steps shown 

p in FIG. 3 (the determining of diagnostic information) are 

jSjj performed during the spinning process described above, so that 

:=J diagnostic information about the object for each of the 

different views in the spinning plane is automatically 
V§ computed. The other steps of FIG. 3 may be performed during 

;3 the spinning process or, preferably, subsequent thereto. 

n*j The volume, geometrical location, and center of mass of 

□ the target object are determined (step 305). The circularity 

or sphericity of the target object is determined (step 310) . 
20 The mean, variance, and min/max of the intensity values within 

the target object are determined (step 315) . The texture, the 
surface smoothness, and regularity measures of the target 
object are determined (step 320) . Two-dimensional and three- 
dimensional shape measures of the target object are determined 
25 (step 325) . 

It is to be appreciated that the diagnostic information 
described with respect to steps 305-325 are merely 
illustrative of some of the diagnostic information which may 
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be determined in accordance with the present invention. It is 
to be further appreciated that any combination or singular 
type of diagnostic information may be determined. Given the 
teachings of the present invention provided herein, one of 
ordinary skill in the related art will contemplate these and 
other types of diagnostic information, while maintaining the 
spirit and scope of the invention. 

The computed diagnostic information gives rise to a 
confidence value (s), also stored in the table, describing an 
estimate of the clinical relevance of the object (step 330) . 
Objects that are considered to be likely to be diseased will 
have a high confidence value. Other objects are that 
considered to be clinically significant, even if not 
indicative of actual disease, may also have a high confidence 
value. An example of the latter is a benign growth. 

The computed diagnostic information of the target 
object and the associated confidence value (s) are displayed 
graphically to the physician (e.g., during or subsequent to 
the display of the cine loop or storage thereof) (step 3 35) , 
and are then stored in a table for comparison with earlier or 
subsequent scans of the same patient (step 3 40) . This 
comparison may be used for change detection in a patient who 
is being screened at some interval, or for monitoring progress 
in a patient undergoing treatment. 

Upon request by the physician, the target object can be 
rendered using standard three-dimensional-volume visualization 
methods. This limited rendering of the target object by 
itself is less confusing and easier to interpret than a 
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standard three-dimensional volume rendering of the entire 
volume data set, where other structures may block the view of 
the object. 

FIG. 4 is a flow diagram illustrating a method for 
suggesting candidate areas of suspicion, according to an 
illustrative embodiment of the invention. 

Based on the part of the body to be evaluated and the 
particular disease to be identified, thresholds are set for 
identifying features of objects that may be suspicious (step 
405) . These thresholds are set as default thresholds, which 
may be overridden and/or adjusted by a physician, as described 
below. Since these thresholds generally result from training 
of the system, and since they are used as default values, the 
thresholds are generally set prior to performing the method of 
FIG. 2. 

Then, for each execution of the method of FIG. 2, the 
thresholds described above may be adjusted by the physician, 
as necessary (step 410) . These adjustments may be made to 
meet individual preferences, or may be adjusted automatically 
in response to additional training data generated by 
physicians.- These thresholds may also be adjusted to target a 
search for a particular disease or condition. 

Next, objects are automatically identified as 
"suspicious", based upon the thresholds (either the default 
thresholds or the adjusted thresholds, if any) (step 415) . 
Because computing object features over the entire data is 
time-consuming, the invention calculates the suspicious areas 
ahead of time, or in the background during the physician's 



manual inspection of the data. 

When a large enough number of target objects have been 
identified, the current data set may be automatically 
registered with one or more data sets taken of the same 
patient at another time(s) (step 420). This registration will 
allow the system to directly compare the characteristics 
stored in the table (as described above) for a given object to 
be compared with characteristics of the same object at a 
different time (step 425) . The comparison is presented to the 
physician so that the physician can monitor the patient for 
development of new disease, or for successful treatment of a 
detected disease (step 430) . 

FIG. 5 illustrates a two-dimensional "slice" from a 
three-dimensional data set obtained by computed tomography 
(CT) of the lungs, according to an illustrative embodiment of 
the invention. The box 510 shows a region of interest (ROD , 
centered on a roughly circular area that looks "suspicious". 
The circular area could correspond to a cross section of 
sphere or a cylinder. Spherical shapes are highly likely to 
be tumors, whereas cylindrical shapes generally correspond to 
branching blood vessels supplying blood to the lungs . 

FIG. 6 is a magnified view of the region of interest 
(ROI) of FIG. 5, according to an illustrative embodiment of 
the invention. FIG. 7 illustrates frames from a "fly-around" 
movie, taken at 15 degree increments of the "spinning plane", 
according to an illustrative embodiment of the invention. As 
the angle approaches 60 degrees, it becomes obvious that the 
circular area in the center of FIG. 6 corresponds to a blood 




vessel and not a tumor. 

Although the illustrative embodiments have been 
described herein with reference to the accompanying drawings, 
it is to be understood that the present system and method is 
not limited to those precise embodiments, and that various 
other changes and modifications may be affected therein by one 
skilled in the art without departing from the scope or spirit 
of the invention. All such changes and modifications are 
intended to be included within the scope of the invention as 
defined by the appended claims. 



